Repository navigation
PQ P5: hybrid key schedule (mlkem_ss || x25519_ss) — first X25519MLKEM768 handshake - #315
Merged
Merged
Conversation
…racle on app traffic secrets, tests and mutants Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…and df_last_err Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ILLED requires a FAIL line (no code byte change) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
PQ phase P5. Under
PQ_HYBRID=1the key schedule now follows the group the ServerHello chose:tls_kex_group == $ECextracts the handshake secret from the 64 Bmlkem_ss || x25519_ss(draft-ietf-tls-ecdhe-mlkem order, ML-KEM first);$1Dis unchanged. After #314 an 11EC server failed at the first encrypted record; now the handshake completes and the GET returns. Both shared secrets are zeroed as soon as the handshake secret exists.Changes:
src/tls_keyschedule.s(the IKM choice and the zeroing, all.ifdef PQ_HYBRID),src/crypto/pq_hybrid.s(outerphp/plpremoved frompq_sh_decaps), the oracletools/pq_keylog_check.py, a new suitetools/test_pq_keyschedule.py,test_pq_kem.py[4] extended, 12 new mutants intools/mutate_pq_sh.py,openssl_listener.py --groups default / --trace, two diag lines in the rig, CLAUDE.md (terse) and engineering-notes.Decisions
tls_mlkem_ssis in NET_BSS_TAIL andtls_shared_secretin data.s BSS, so the 64 B IKM is copied intoinput_buffer(dead scratch since the early-secret step) and the HKDF IKM pointer/length retargeted.hkdf_extractcopies the IKM intohmac_data_buf(97 B) and HMACs it as the message (the salt is the key), so 64 is within its limit.tls_shared_secret,tls_mlkem_ss,input_buffer[0:64],hmac_data_buf[0:64]. Later HKDF calls reusehmac_data_buf, so the test checks it by content (no 8-byte window of the IKM survives); mutant K11 shows bytes around offset 55-63 do survive without the zeroing. 'Q' spans unchanged and still correct (the seed+ss span just holds zeros earlier).tls_ecdhe_privkeystays until 'Q' (same as classic builds); SHA-256's internal block/schedule buffers were not inspected (the next HMACs of the same routine rewrite them).pq_keylog_check.py --statenow comparestls_c_hs_secret/tls_s_hs_secret(reused for the application traffic secrets aftertls_derive_traffic_keys) with the keylog'sCLIENT_TRAFFIC_SECRET_0/SERVER_TRAFFIC_SECRET_0. Those are a pure function of the hybrid handshake secret and the transcript through the server Finished. Gate:tls_reached_connected == 7(tls_state has no high-water latch, so the rig's CONNECTED oracle is outrun at turbo #204),tls_kex_group == $EC,tls_sh_err == 0, secrets non-zero; any non-zero seed/ss/shared-secret in a connected run is a RESIDUE failure (exit 1). The explicit-argument P4 mode is unchanged. PQ_DUMP now also dumps the two secrets and the latch.pq_sh_decaps: redundant outerphp/plpremoved (pq_mlkem_scrubsaves/restores P;pha/plakept). Mutants Y3p/Y4d still kill the scrub's own save/restore.Knob-off PRGs byte-identical to master 4e45a9b (sha256, built before/after on this tree:
git checkout 4e45a9bthen the branch)(
make/USE_NISTCURVES_ONCHIP=1/BACKEND=uci/+USE_NISTCURVES_ONCHIP=1/+USE_NISTCURVES_ONCHIP_COMB=1/+TRUST_STORE=1variants;make clean; sleep 1between builds. Equal to the #314 table.)PQ images (non-SNI), before (4e45a9b) -> after
Margins (
python3 tools/measure_margins.py --build --profile uci-onchip-pq --profile uci-comb-pq --profile uci-onchip-store-pq --profile uci-comb-store-pq, free bytes, #314 -> now)Cost: about 63 B of TLS_CODE (NET_CODE), 2 B back from the
pq_sh_decapscleanup (CRYPTO_HOT on comb). The tightest region on uci-comb-store-pq is CRYPTO_OVERLAY (55 B, unchanged); NET_CODE on the store profiles is down to 105-134 B. Comb loses 63 B of NET_CODE, onchip 61 B.Red -> green
git checkout 4e45a9b -- src/tls_keyschedule.s src/crypto/pq_hybrid.s; python3 tools/test_pq_keyschedule.py):RESULTS: 28/48 passed, 20 failed. Every$ECcheck against the host HKDF fails (handshake secret, both traffic secrets, 4 key/iv buffers, twice), the secrets are not zeroed; the$1Dand group-0 schedule checks pass on master (the classic path).python3 tools/test_pq_keyschedule.py: 48/48. The fixture plantstls_mlkem_ss,tls_shared_secret, group and transcript, runstls_derive_handshake_keys, and compares withhashlib/hmacHKDF over the host-known IKM; the host references for hybrid / classic / swapped-order differ (asserted).tools/test_pq_kem.py[4] (tls_connectend to end) also asserts, at a breakpoint ontls_derive_handshake_keys, that the two secrets and the transcript are the host's, and afterwardstls_handshake_secret== host HKDF overK || ss(11EC) /ss(001D), both secrets zero.tools/test_pq_keylog_check.pyred cases per input (either secret, swapped client/server, residue in each of seed/ss/shared secret, missing keylog lines, 7 gate arms incl. "CONNECTED then closed reads tls_state=0" green and "ERROR with a stale latch" green).Tests (all on this head, exact commands)
python3 tools/test_pq_keyschedule.pypython3 tools/test_pq_sh.pyC64_SKIP_BUILD=1 python3 tools/test_pq_kem.pypython3 tools/test_mlkem_placement.pypython3 tools/test_mlkem_linked.py(with mutants)python3 tools/test_pq_keylog_check.py/python3 tools/pq_keylog_check.py --selftest/python3 tools/test_openssl_listener.pyBACKEND=uci python3 tools/test_hs_sequence.pyBACKEND=uci python3 tools/test_tls_handshake.pypython3 tools/test_keyschedule_steps.py/test_hkdf.py/test_finished_verify.pypython3 tools/test_runner_coverage.py/test_pytest_boundary.pypython3 tools/test_build_flags_stamp.pypython3 -m pytest -qMutants (
python3 tools/mutate_pq_sh.py --no-control K01..K12 M17 M23 M43 M44 M49 Y3p Y4d: 19/19 killed, control after restore green; each dies by an assertion)bne->bcc)mutate_pq_sh.pynow reports a non-zero exit without an assertion FAIL line as CRASH-OR-HANG (not a kill). The full 56 P4 mutants were not re-run (only the ones whose code I touched);tools/mutate_pq_sh.pycontrols now include the new suite.Hardware (U64E 10.43.23.81 via DeviceLock, 48 MHz, OpenSSL 3.6.5
s_server, free ports 24510-24512, 4433 untouched)Local listener =
tools/https_e2e/openssl_listener.py+tools/uci/rig_https_local.pywithEXTERNAL_LISTENER=1 EXTERNAL_PORT=<p> PQ_DUMP=<dir>; thenpython3 tools/pq_keylog_check.py --state <dir>/pq_state.json --keylog <dir>/keylog.txt. "OpenSSL group" = the ServerHello key_share group ins_server -trace.Images (both built from d226402's src; later commits are tools/docs):
make BACKEND=uci USE_NISTCURVES_ONCHIP=1 PQ_HYBRID=1 HTTPS_SNI=www.foo.invalidsha256ab85014cdc8bb074fc9cb99180c90dc8a74ac4d8ed9f68cfda32e7d311cf5e0fC64_INIT_WAIT=90):make BACKEND=uci USE_NISTCURVES_ONCHIP_COMB=1 PQ_HYBRID=1 HTTPS_SNI=www.foo.invalidsha256265c4c583c2b5a2430daf3a7ee2a3e546d99a8b7c1c4173476e8455097cfa042-groupsHELLO FROM TLS SERVER)-groups)With
PHASE_TIMING=1(comb, 1 s poll, n=1 each): trigger to client Finished 43.7 s hybrid vs 42.0 s X25519-only on the same image, i.e. about +1.7 s (~4%) for KeyGen + Decaps + the bigger flight. Rig wall-clock above includes boot/prep and is within 1-2 s between groups.boot_check.pyPASS on both images.Round 1, store path:
make BACKEND=uci USE_NISTCURVES_ONCHIP=1 TRUST_STORE=1 PQ_HYBRID=1 HTTPS_SNI=www.foo.invalid, sha25646398f0bd258de344f75f2df8994b7bbf8c913d1a59cd7054db5442c4e43d400, local-groups X25519MLKEM768(port 24513): full HTTPS PASS, OpenSSL group 11EC, rig 107 s, oracle MATCH (cert_pin_hs_keys -> tls_derive_handshake_keys).rig_https_local.pyneeds no trust files (its trampoline sets tp_mode, tools/uci/_trust_mode.py).Round 1 no-code-change proof: the 7 knob-off hashes, the 4 non-SNI PQ hashes and both SNI hardware hashes (ab85014c..., 265c4c58...) rebuilt identical after the comment/test changes. Docs now carry the image hashes, make lines and timings (engineering-notes P5 section); CLAUDE.md distinguishes local (onchip+comb) from live (comb only; wikipedia = HTTP 301 at
/). OpenSSL 3.6.5 with no-groupspicks X25519MLKEM768 for our hello.Bonus, real public servers (
tools/uci/rig_https_live.py, comb image withoutHTTPS_SNI:make BACKEND=uci USE_NISTCURVES_ONCHIP_COMB=1 PQ_HYBRID=1, sha256b8677dd79968fb57e475baaa0995c8979d670b159ae753b207a947d5e8be8d61):tls_kex_group=$EC, HTTP 200,tls_reached_connected=7, client Finished at 44.1 s (the non-PQ comb imagead85797b...: 42.3 s).EXPECT_STATUS=301,/): PASS,$EC, HTTP 301, 39.8 s.openssl s_client -groups X25519MLKEM768getshandshake_failure(no 11EC); not run on the C64.HTTPS_SNI=www.foo.invalidfails a LIVE host withdf_last_err=$05(SAN checked against the baked name). It looked like a hybrid defect for one run; the same image without the override passes.Known gaps
test_pq_keyschedule.pyis what pins the IKM order.$(error)for PQ_HYBRID.🤖 Generated with Claude Code